Person-to-person communication in meetings having routine discussions, brainstorming sessions, or formal presentations are often characterized by redundant or ill-defined verbal and written expressions that may hamper comprehension or reduce the efficiency of the information exchanged by the meeting participants.
During meetings, for example, people present information to each other across multiple modes. Graphically, they may sketch diagrams, like a schedule chart or timeline. Textually, they may handwrite lists of preferred points or concepts, they may label parts of a diagram, or they may type information real time for display on a display screen. While sketching or handwriting they are also likely speaking to each other. Speakers may handwrite on public surfaces (like whiteboards, flipcharts or even table napkins), while listeners jot down personal notes on paper.
People in interaction are always creating new vocabulary. Computational systems with fixed recognition vocabularies cannot recognize such new vocabulary. In order to be better able to understand natural interactions, computational systems need to be able to learn new vocabulary dynamically as they perceive natural communications. For example, the Defense Advanced Research Projects Agency's Cognitive Assistant that Learns and Organizes (CALO) attempts to provide at least some learning capabilities that may eventually support artificially intelligent systems for responding robustly to surprising or unforeseen inputs, just like people do. The CALO project has been attempting to transform computational systems from being simply reactive to being more cognitive.